Black-box Landscapes: Characterization, Optimization, Sampling, and Application to Geometric Configuration Problems

In many areas of science and engineering researchers consider systems that can be solely examined by their input and output characteristics without any knowledge of their internal workings. Such black-box systems are the topic of the present thesis. In many practical cases, a black box comprises a complex mathematical model, a computer simulation, a real-world experiment, or a combination of any of these. In this thesis we take an interdisciplinary approach to the characterization, optimization, and sampling of black-box systems. We focus on systems with high-dimensional real-valued input variables and output patterns that can be transformed by some function into a scalar real-valued quantity. Throughout this thesis we conceptualize the black-box system as a landscape. Inspired by our shared visual experience of natural terrains and sceneries, we consider the real-valued input variables as a high-dimensional landscape domain. Neighborhood or nearness in this landscape domain must be provided by a suitable distance metric. We interpret the scalar output quantity as a height or elevation over the landscape domain. The landscape metaphor encourages a characterization of blackbox systems in terms of topographical features, such as valleys, ridges, mountain peaks, and plateaus. In order to underline that we view black-box systems as high-dimensional, complex landscapes we introduce the notion of the black-box landscape. After a general review of the landscape paradigm, spanning the disciplines of biology, physics, chemistry, and optimization, we present a number of statistical landscape descriptors that probe different properties of black-box landscapes. The core of the thesis is concerned with black-box optimization. We improve the performance of the arguably best state-of-the-art optimizer, the Covariance Matrix Adaptation Evolution Strategy (CMA-ES), in various aspects. The general performance is increased by considering quasi-random instead of pseudo-random sampling. For multi-funnel landscape topologies we introduce parallel CMA-ES schemes that can outperform standard CMA-ES. We also revisit Gaussian Adaptation, an optimization and sampling scheme that has been largely ignored in the black-box optimization community. Our improved Gaussian Adaptation scheme shows remarkable performance on the considered benchmarks and ranks among the best known black-box optimizers. An important conceptual result is that we can provide an explicit link between black-box optimization and black-box (or indirect) sampling through Gaussian Adaptation. We show that the same idea of adaptation has emerged in these disparate fields, and we argue that a unifying framework for sampling and optimization might constitute an important contribution. We further consider geometric configurations in two different contexts: Geometry optimization problems of atomic clusters are proposed as novel benchmarks for black-box optimization. We design a balanced set of problems that should be included in future black-box optimization benchmarks. We also revisit the configuration space of chain molecules with respect to a certain distance measure, the Root Mean Square Deviation (RMSD) after optimal superposition. Because RMSD is the most important distance metric